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VIII 



In many scientific endeavors, researchers want to . determine 
whether a sequence of observations taken over time exhibits some 
type of trend. The NASA Standard, "Trend Analysis Techniques^ 
(NASA-STD— 8070 . 5) , describes a variety of statistical methods that 
could be applied to time series data. Generally, trend is regarded 
as a smooth broad motion of the system over a "long" term. Several 
techniques are currently being used. The result of these efforts is 
reported in "Quarterly Problem Trending Report for MSFC Shuttle 
Elements and Payloads" prepared by the Calspan Corporation. 
However, the nature of the problem/failure data poses difficulty in 

identifying a trend. . . . . . _ _ 

One of the major obstacles in identifying and interpreting a 
trend is the small number of data points. Future trending reports 
will begin with 1983 data. As the problem/ failure data is 
aggregated by year, there are just seven observations (1983-1989) 
for the 1990 reports. Any statistical inferences with a small 
amount of data will have a large degree of uncertainty. 
Consequently, a regression technique approach to identify a trend 
is limited. Though trend determination by failure mode may be 
unrealistic, the data may be explored for consistency or stability 
and the failure rate investigated. In what follows, various 
alternative data analysis procedures are briefly discussed. 
Techniques that could be used to explore problem/ failure data by 
failure mode. The data used is taken from Section One, Space 
Shuttle Main Engine, of the Calspan Quarterly Report dated April 2, 


1990 • 

There were four set of observations in the Quarterly Report 
SSME Section that had a statistically significant downward trend 
based on a regression analysis. There were a total of 36 data sets 
trended in the SSME Section. These significant trends were based on 
data from 1979 through 1989. Reconsidering these sets of data from 
1983 on, not one of them has a significant regression fit at the 
.01 level, which is the level used in the Quarterly Report. If we 
begin with data in 1983, a significant downward trend at the .01 
level of significance requires a Pearson product moment correlation 
coefficient of r= -.875, that is r 1 = .765. In addition to the 
Pearson correlation coefficient with the corresponding test of 
hypothesis based on the normal distribution, there are numerous 
nonparametric measures of association. Two widely used ones in 
connection with regression and trending are Spearman's rank 
correlation coefficient and Kendall's tau. These two procedures use 
the rank order of the observations rather than the actual observed 
value. The four sets of data that were previously fit with a 
regression model are summarized below with the data beginning in 
1983. 
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Correlations and Observed Significance Levels 


Fuel preburner injector 
erosion/wear 

dents/etc 

contamination 

Controller hardware 
unexplained anomalies 


Pearson Spearman 


-. 618 (linear) 

-.607 

p=. 139 

p= . 137 

-.765 (power) 

-.857 

p=. 047 

p=. 036 

-.675 (linear) 

-.685 

p = .096 

p= . 094 

-. 805 (linear) 

-.786 

p=. 029 

p=. 054 


Kendall 

-.429 
p= . 177 
-.714 
p= . 024 
-.488 
p=.lll 

-.714 
p= . 024 


Another approach to exploring the data is from the 
perspective of consistency. That is, does the failure rate 
fluctuate from year to year or is it relatively stable? While this 
approach is not to identify a trend it may provide the 
experimenter with insight into the failure process. Assuming a 
Poisson model for the number of failures, a chi-square goodness of 
fit test, assuming a constant failure rate from 1983-1989, a 
Kolmogorov-Smirnov goodness of fit test with the same assumption, 
or a level of performance chart could be applied. 

The chi-square goodness of fit test utilizes an assumed 
probability model and estimates the expected number of failures 
per year and then compares the observed number of failures with 
these expected frequencies. To specify the procedure, let be 
the number of failures in year i and let t^ be the number of test 
seconds (or test starts) in year i. The expected number of 
failures, under the assumption of a constant failure rate, for 
year i is given by /Lt± = et-^, where 6 is the probability of a 
failure in a smjall interval. The constant failure rate estimate of 
8 is given by 6 = Z X^/Z t^, where the summation ranges through 
the years 1983 to 1989. The chi-square test statistic is given by 
T= Z(Xj-/2i) 2 /Mi which is approximately distributed as a chi-square 
random variable with 5 degrees of freedom. The table below displays 
observed and expected frequencies for the 3 fuel preburner injector 
failure modes plus 2 other randomly selected data sets. 

The .01 critical value for a chi-square distribution with 5 
degrees of freedom is 15.1. The value of the test statistic T 
gives some idea of the agreement between the observed results and 
the assumption of a constant failure rate. The value of T must be 
cautiously viewed in cases of small expected frequencies. Consult 
the reference by Lawal and Upton for this consideration. 
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Observed and Expe cted Frequencies 



83 

84 

85 

86 

87 

88 a 2. 

Fuel preburner injector 
erosion/ wear 

3 

2.73 

4 

2.52 

5 

3.83 

1 

1.21 

2 

3.95 

4 3 

4.30 3.5 

T~2.33 

dents/etc 

13 

3.72 

3 

3.44 

7 

5.23 

1 

1.65 

1 

5.39 

3 2 

5.86 4.7 

T=30 . 60 

cont am i na t i on 

3 

1.86 

5 

1.72 

4 

2.61 

0 

.82 

1 

2.70 

0 2 
2.93 2.4 

T=12 . 57 

LPOTP contamination 

6 

6.45 

10 

5.96 

2 

9.06 

3 

2.85 

15 

9.35 

13 3 

10.2 8.2 
T=15 .76 

HPOTP turbine nozzle 
second stage crack 

1 

1.61 

1 

1.49 

1 

2.27 

1 

.71 

0 

2.34 

7 2 

2.54 2.0 

T=ll . 39 


Another technique to explore the stability of the failure 
rate is a level of performance chart. It is constructed and used 
as a control chart. Using the Poisson probability model, the 
overall f g i lure rate estimate © is used to compute limits of 
6 + 37$ /Z ti . The yearly failure rate estimates ©i are then 
compared, often graphically, with these limits. Values of ©i 
outside the limits point to extreme fluctuation of the failure 
rates. The table below gives the yearly estimates of 0 with limits 
based on the pooled estimate for 3 of the failure modes previously 
considered. 


Estimates of © bv Year ( x 10~1) 



83 

84 

85 

85 

87 

88 

89 

Limits 

Fuel preburner injector 
erosion/wear 

ac ac • 

.10 

.15 

.12 

.08 

.05 

.09 

.08 

(.03, .16) 

dents/ etc 

.45 

.11 

.17 

.08 

.02 

.07 

.05 

( .06, .20) 

HPOTP turbine nozzle 
second stage crack 

.03 

.04 

.02 

.08 

0 

.15 

.05 

(.01, .10) 


An alternative process control technique that would monitor 
the yearly failure rate relative to a specified target failure 
rate is a cumulative sum (CUSUM) procedure. The CUSUM procedure is 
a sequence of Wald sequential probability ratio tests used to 
detect a change in the distribution of the number of problems. As 
before, a Poisson distribution for the number of problems is used. 
The CUSUM procedure is often enhanced by. a fast initial response 
(FIR) feature. The reference by Lucas discusses a Poisson CUSUM 

^ A cumulative sum procedure cumulates the difference between 
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an observed value Y^, some normalized value of Xj_, and reference 
value k. If this cumulation equals or exceeds the decision value 
h, then conclude the failure rate for that year is greater than 
the target rate. To detect an increase in counts, the CUSUM 
statistic is Sj[= max(0, Yi-k+S^.!) . The FIR CUSUM typically uses a 
starting value S 0 = h/2. The CUSUM is restarted after indicating an 
out of target value. 

CUSUM procedures are evaluated by calculating their average 
run length (ARL) . The ARL should be large when the failure rate is 
at the target level and short when the rate is at an undesireable 
level . 

The parameter k is the reference value for the CUSUM. Its 
value will be chosen to be between the acceptable failure rate 
(M a ) and an unacceptable level (m<j) that is to be detected. 
Although the desired level of n a is zero, it is usually not used 
since any occurence of a failure will then give a signal. The 
reference value for the Poisson CUSUM should be selected to be 
close to k= (Mji -n a )/(ln Md _ln M a )• 

After k is selected, the decision value h is chosen using a 
table look-up procedure. There are tables given in the article by 
Lucas. The value of h should give an appropriately large ARL when 
the failure rate is on target and an appropriately small ARL value 
when the rate is too high. 

Use of a CUSUM takes more involvement from the analyst than 
do the goodness of fit test or the level of performance chart. The 
CUSUM, however, combines looking at the data for stability and 
checking agreement with a target value. A CUSUM procedure with an 
acceptable failure rate of 1 per 50,000 seconds and an 
unacceptable rate of 1 per 20,000 seconds has been applied to the 
problem/ failure data. When a CUSUM value exceeds h then it is 
restarted with next data value. The results are comparable to 
those from the goodness of fit test and rate performance chart. 
Along with the chi-square goodness of fit test and the level of 
performance chart, even though simple, these techniques offer some 
insight to supplement the regression approach. 

Trend fitting and trend estimation are very far, particularly 
with small samples, from being a purely mechanial process. There is 
great scope, even necessity, for personal judgement. Exploring the 
data for patterns can be a very difficult, delicate issue. 
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